1. Field of the Invention
The present invention relates to an image observation apparatus, and more particularly, improvement of an image observation apparatus such that it can be easily performed for a small scale apparatus to create three-dimensional (hereinafter, 3D) images and process images of games and so on.
2. Related Background Art
FIG. 44 among drawings incorporated in this application shows an example of work flows for creating motion pictures with using 3D images. First, object data (basic data) is created at process P1, and at the next process P2, an animation work, that is, a motion picture creation work is performed. Subsequently, at process P3, a final rendering work, i.e., a complete motion picture creation work is performed. Motion picture data created at process P3 is recorded on a recording medium or media at process P4 as a recording work. These works are common in this field.
For example, when 3D images are created, the works in the above-mentioned object data creation process (process P1 are shown in FIG. 49. The works contain providing two image-sensing apparatus (camera L and camera R) in line, taking an image of a subject with each camera, and recording a pair of image L and image R in respective recording apparatus (video deck L and video deck R). Then, this recorded pair of images becomes animation data of a live-action having a binocular parallax.
On the other hand, for example, as the works in the object data creation process, two works are performed. The first work of them is to create a pair of images, one image to be played back for a left eye and the other to be done for a right eye, every frame by a computer and the like respectively (so-called computer graphics (CG)). This produced pair of images is made to be animation data, and the next work is to create a final animation on the basis of this data. Further, the pair of animation images having a binocular parallax is recorded on separate recording apparatuses respectively.
With further elaborating, for example, as a procedure for producing 3D images in the CG, first, in an object data producing process P1 shown in FIG. 44, the image data regarding the fundamental skelton of the object is produced in rough polygons, that is, a small number of polygons. In addition, simple color data is also set.
Next, animation data regarding the image data produced in the above-mentioned process P1, is produced in the animation work performed in the process P2. In this work, popular application software for edition of animation and the like are used. After completion of this edition of the animation, the final rendering work is performed in the next process P3.
In the process P3, the final rendering work is performed every screen frame. Thus, with forming a minute shape with a plenty of polygons, setting of proper color data such as texture mapping is performed.
The image data completed in this manner is recorded in the recording work in the process P4. That is, the image data is outputted from a computer for edition to a recording apparatus such as a video deck conforming to a common standard, for example, VHS or xcex2-cam, the image data which is recorded on a recording medium such as a video recording tape by the recording apparatus.
In addition, when a 3D image is produced with using live-action data, its procedure is as follows.
First, it is performed to sense the images of an object. With using two video camera recorders (VCRs) and the like, an image for a right eye that should be played back to only the right eye (hereinafter called an R image) and an image for a left eye that should be played back to only the left eye (hereinafter called an L image) are sensed respectively at the same time.
These images sensed in this manner are played back to left and right eyes respectively with synchronizing the R image and L image with each other. Further, by confirming 3D effects such as an offset quantity (described later), it is checked whether proper 3D images could be sensed. Thus, this confirms whether proper 3D image-sensing was performed. Then, until the recorded images become proper ones, after revising, for example, a sensing position of the VCR, the distance between the VCR and object, and the like, the processes of image-sensing and playback check are repeated. Although these processes are the work corresponding to the process P2 shown in FIG. 44, it is possible to produce a 3D image in only the above-mentioned processes of the image-sensing and playback check in case of live-action data.
Here, 3D images using a binocular parallax will be described below with reference to FIGS. 45-48 among drawings incorporated in this application.
An image for a left eye, referred to as code L in FIG. 45, and an image for a right eye, referred to as code R are images of object data taken or created by supposing the case that a single object, shown as a round object in FIG. 46 is observed by observer""s left and right eyes respectively.
If the image for a left eye, L and the image for a right eye, R are played back independently to observer""s left and right eyes and are displayed on display means such as an LCD, observer""s left and right eyes, as shown in FIG. 46 look at a virtual image screen of display means (LCD etc.) in a playback apparatus. However, in this time, object data on the virtual image screen of the display means, that is, two object data of the image for the left eye, L and the image for the right eye, R can be observed as a single object by being recognized as an identical object (this phenomenon is called xe2x80x9cfusionxe2x80x9d).
In addition, in order to enable an object to be observed as if it protruded in this side, i.e., to enable the object to be observed three-dimensionally, a pair of images L and R of object data are taken or created, the pair of images being composed of an image for the left eye, L and an image for the right eye, R, as shown in FIG. 47. In this case, positions of the two object data composed of the image for the left eye, L and the image for the right eye, R are mutually reversed on a virtual image screen of display means (LCD etc.) as shown in FIG. 48. Owing to this, the object image being recognized as an identical object by observer""s eyes is observed as if the object protruded in this side by an offset quantity shown in FIG. 48.
Furthermore, heretofore, taking or creating object data during above-mentioned works for creating 3D images is composed of the following works. The first work is to provide two independent cameras, that is, a camera to take an image for the left eye, L and another camera to take an image for the right eye, R as shown in FIG. 49, in line to a subject. The second work is to record a pair of images (an image for the left eye, L and an image for the right eye, R) taken with respective cameras L and R, in two independent recording apparatuses, that is, a video deck L for recording the image for the left eye, L and a video deck R for recording the image for the right eye, R. These images are recorded on separate recording media, which are used as basic data (animation data having a binocular parallax).
Moreover, reference codes La and Ra shown in FIG. 49 show angles of view of image pickup optical systems for cameras L and R, i.e., ranges where images can be taken, and the angle of view of the camera L, La and the angle of view of the camera R, Ra are set to become approximately equal and to overlap each other. In addition, by providing this overlapped part, 3D sense to a subject included in this range can be obtained.
Furthermore, conventionally, confirmation of 3D images of the object data (basic data) taken or created, and recorded as described above is performed as follows.
Thus, as shown in FIG. 50, for example, the above-mentioned confirmation of the 3D images at the time of creating the images is performed by playing the image for the left eye, L and the image for the right eye, R simultaneously back with two individual video decks L and R keeping synchronization, and outputting respective image signals to two video camera recorders (VCRs) each having a recording and playback function, L and R. Furthermore, respective images L and R are displayed separately for observer""s left and right eyes on finder parts for confirmation of the 3D images which are observation means and display means, both of which are provided in these video camera recorders L and R, and each comprise a display unit such as a liquid crystal display (LCD) and a CRT.
In this manner, it is requested to advance production with confirming every time the 3D images are taken or created in the process of creating the object data (P1), the animation process (P2), the final rendering process (P3) and so on as described above in FIG. 44.
There are various items to be confirmed at the time of image production such as what extent of 3D sense is adequate, whether verve is sufficient, and whether the pair of left and right images has parallax to disable images to fuse (excessively large parallax). Further items are whether the balance of offset amounts in the longitudinal direction is adequate among images in former and latter scenes when scenes are connected by editing respective images, and so on.
The above-mentioned requirement, that is, the requirement to advance works with confirmation at the time of image production becomes stronger in the case of using the 3D images to be produced in games, amusements, entertainment, etc., and in particular, in the case of desiring to obtain images having enough presence.
On the other hand, various types of image-sensing apparatuses (systems) for taking 3D images have been proposed.
For example, the 3D image-sensing apparatus disclosed in Japanese Unexamined Patent Publication No. 7-75134 comprises a camera apparatus composed of a pair of left and right small cameras, a controller for controlling the position of this camera apparatus, a monitor unit for displaying images taken with the camera apparatus respectively, etc.
In addition, the monitor unit is provided with a pair of small monitors respectively displaying the images from the pair of left and right small cameras, and eye-cups internally having lenses to enlarge the images taken with the camera apparatus and to enable an observer to observe 3D images.
Furthermore, the camera apparatus can be moved in the up and down directions and the left and right directions by the controller, and hence, an observer can easily observe images in the desired direction by controlling the position of the camera apparatus with the controller.
On the other hand, recently, so-called computer games, each of which is run by displaying a motion picture on a screen of a TV set, or a display unit of a personal computer (PC) and the like (hereafter, a display and the like) have been in the market.
These games have various types of content. For example, as shown in FIG. 51, a game has a long screen frame, a displayed screen scrolls vertically (Y arrow directions shown in FIG. 51), and the game proceeds, for example, by shooting down enemy fighters in turn. Another game, as shown in FIG. 52 has a wide screen frame, a displayed screen scrolls horizontally (X arrow directions shown in FIG. 52), and the game proceeds, for example, by falling enemy characters in turn.
Therefore, conditions such as an optimal screen frame size, and an aspect ratio of the screen are different every game, the conditions being not only for most enjoying each game but also for producing better comfortableness and presence.
In addition, in a game where emphasis is put on speed sense, the game can be sufficiently enjoyed even if CPU""s processing is set for speed to acquire the priority and quality of display images is set low. However, in a game where emphasis is put on contents of the game (story) and/or display images of game characters including motion pictures themselves, it is better for further enjoying the game to set the display images at precise images.
Then, when each of various types of game software is created, it is desirable to use a display and the like for confirming images, which are similar to the display and the like for playing the game, regarding a screen frame size, an aspect ratio, picture quality, etc.
However, presently, in case of home video games that are enjoyed personally at usual home, it is common to use home TV sets as displays and the like for them. Owing to this, it is usual that the screen frame size, aspect ratio, picture quality and the like cannot be set and changed in conditions suitable to contents of various types of games. Therefore, various types of current game software are produced with scarcely considering change of various parameter settings.
On the other hand, in order to make it possible to enjoy any types of games, a dedicated display and the like, having a high resolution screen and a large screen frame, may be provided. However, since this type of display is extremely expensive and also requires a broad space, this is not practical.
Above problems are common not only at home, but also in amusement parks, gaming centers, and development sites of game software.
Then, recently, head-mounted types of display units such as so-called head-mounted displays (HMDs), each of which is mounted on observer""s head with the display covering from his/her head to his/her face, have been proposed as small-sized displays for displaying images and using for various types of games.
For example, a display for games, disclosed in Japanese Unexamined Patent Publication No. 4-33679 comprises an image projection unit and a light path changing member (mirror) arranged for directing image display light, projected from this image projection unit, toward a predetermined direction. Furthermore, the image projection unit comprises: optical image display means having a display screen to display images corresponding to image signals from the main frame of a game machine; and an optical member to project the image display light projected from the display screen toward the light path change member (mirror).
According to this, it becomes possible to be absorbed in the world of games (atmosphere) since surrounding unnecessary information can be cut off. Moreover, it is also possible to reduce eye fatigue and lowering of vision by adaptively keeping the distance between eyes and display images.
In addition, a shooting toy, disclosed in Japanese Unexamined Patent Publication No. 5-180595, comprises a mock gun, a goggle, and a cable transmitting signals between the mock gun and the goggle. Its control means judges hitting on the basis of a trigger signal from the mock gun and the status of a signal received by infrared ray receiving means. Meanwhile, the control means judges that the game player is hit when the infrared ray receiving means of the goggle receives a predetermined infrared signal, and then, the control means makes the display means display information of hitting, being hit, etc. on a part of glass in the goggle.
However, as described above, during producing the 3D images, if two separate image-sensing/playback apparatuses are used, the confirmation and revision of the once-recorded images must be performed by operating the two separate image-sensing apparatuses simultaneously, and hence, this toy has a problem that this requires time and labor for performing the creation work of the object data.
Furthermore, in order to take an image similar to an object image when an observer looks at the object with naked eyes, it is necessary to arrange two cameras L and R so that the pupil distance, that is, the distance between both eyes may be approximately equal to the space between two cameras L and R (base-line length; normally, the base-line length is set nearly 50-70 mm and the average length is 65 mm) for taking the image. Furthermore, it is also necessary to install two cameras L and R so that the installed angles of two cameras L and R may meet with the angle of convergence of eyes, when the observer looks at the object image with naked eyes, that is, the angle within which observation ranges of left and right eyes overlap each other.
Obtaining such fine 3D images requires many setting items of image-sensing conditions regarding the arrangement of two cameras L and R for sensing images. Hence, this toy has another problem that, if two image-taking cameras are separate, setting of the image-sensing conditions becomes especially complex.
In addition, if the 3d images in production are confirmed, the finder parts of two video camera recorders L and R separately located are arranged, as conventionally performed. However, this has still another problem that it is extremely difficult to achieve the arrangement due to mutual physical interference of both arranged finder parts.
Furthermore, this has a further problem that observing or confirming the 3D images with a conventional liquid crystal shutter (hereinafter, an LC shutter) type of eye glasses requires a special work of recording the 3D images L and R again for converting to field-sequential images, and hence, this is inefficient due to consumption of time and cost.
Still more, a system, comprising cameras, video decks, etc. (refer to FIG. 50), for observation (confirmation) of the 3D images is composed of the system which is different from units for production of the 3D images such as a computer. Therefore, for confirming the 3D images produced by this computer and the like, it is necessary to move from the installed position of the computer and the like for production of the images to the display units fixedly installed at the different position, that is, the finder parts of the cameras L and R (refer to FIG. 50). Hence, this has an additional problem that working efficiency decreases.
Furthermore, since this requires two image-sensing apparatus, the entire system becomes large, and hence, this has another problem that it is difficult not only to keep a location for installing the system including two image-sensing apparatus, but also to carry and use the system.
Moreover, according to means disclosed in Japanese Unexamined Patent Publication No. 7-75134, its structure becomes large similarly to that of the image-sensing apparatus exemplified in FIG. 49 and the observation system exemplified in FIG. 50. Therefore, it is difficult to carry and use this apparatus, and hence, it seems that object data can not be easily created using this apparatus.
In addition, if it is performed to create and process an image for the left eye and an image for the right eye separately using two separate computers, it becomes necessary to take a pair of left and right created images with two image-sensing apparatuses. Therefore, this apparatus may become a larger system.
On the other hand, a head-mounted display and the like disclosed in the above-mentioned Japanese Unexamined Patent Publication No. 4-33679 and Japanese Unexamined Patent Publication No. 5-180595 do not display information about adequate screen frame and the like corresponding to the contents of various types of games. However, these displays are dedicated ones to respective games, and hence, in these specific games, these displays can be used optimally. However, if they are used in other games, presence in the games may be insufficient, and comfortableness of playing the games may be reduced.
In addition, a head-mounted display can be easily used as a small display for playing various types of games, a display for confirming images at the time of creating various types of game software, and the like. However, since it is impossible to see the surrounding status with wearing this type of head-mounted display, it is difficult and complex to perform various settings of this display itself such as adjustment of the pupil distance and adjustment of visibility, etc. This is another problem.
Furthermore, if a game player does not understand or does not know how to operate the game to be started, the player can not refer to its operation manual with wearing this head-mounted display, which is an additional problem.
Still more, if an unexpected accident such as power failure when a game is being played, the game in progress will be interrupted at that time. Therefore, the progress status and results of the game just before the power failure may be deleted without being stored in a recording medium such as memory.
It is a first object of the present invention to provide an image observation apparatus that not only can easily confirm 3D images when the 3D images are created, but also can display the images on a large screen, and further that is small and has superior portability.
In addition, it is a second object of the present invention to provide an image observation apparatus that can be easily set and modified (for example, on an aspect ratio, picture quality, etc.) when display images of a game and the like are displayed.
Further more, it is a third object of the present invention to provide an image observation apparatus that can be easily set on the apparatus itself such as adjustment of a pupil distance and adjustment of visibility even if a head-mounted display is worn.
Still more, it is a fourth object to provide an image observation apparatus that does not require an operation manual after wearing a head-mounted display by displaying the operation method of a game to be started on the image display unit of the image observation apparatus.
Moreover, it is a fifth object of the present invention to provide an image observation apparatus that can securely protect the progress status and the like of a game when power supply to the apparatus is interrupted during using the apparatus for a game and the like.
In short, an image observation apparatus according to the present invention is characterized in comprising: a head-mounted image display unit (HMD) for displaying images, derived from video signals supplied to itself, by an image display unit corresponding to observer""s eyes; a computer having specifications suitable to the production of images displayed on the head-mounted image display unit, and including an output part for outputting an video signal expressing the images; and a controller being inserted in a signal transmission line between the computer and the head-mounted image display unit and performing operator""s optional operation regarding supply aspect of the video signals from the computer to the head-mounted image display unit.
Therefore, the head-mounted image display unit of the image observation apparatus according to the present invention displays images, derived from video signals supplied to itself, by the image display unit corresponding to observer""s eyes; the computer produces images to be displayed by the head-mounted image display unit and outputs the video signal expressing the images from the output part; and the controller is inserted in the signal transmission line between the computer and the head-mounted image display unit and performs operator""s optional operation regarding supply aspect of the video signal from the computer to the head-mounted image display unit.
In addition, an image observation apparatus according to the present invention is characterized in comprising: a head-mounted image display unit for displaying images, derived from video signals supplied to itself, by an image display unit corresponding to observer""s eyes; and a controller being capable of selectively mounting a recording medium storing image data having a binocular parallax, performing operator""s optional operation to image data read from the mounted recording medium, and supplying a video signal, corresponding to the image data performed the operation, to the head-mounted image display unit.
Therefore, the head-mounted image display unit of the image observation apparatus according to the present invention displays images, derived from video signals supplied to itself, by an image display unit corresponding to observer""s eyes; and the controller can selectively mount a recording medium storing image data having a binocular parallax, performs operator""s optional operation to image data read from the mounted recording medium, and supplies a video signal, corresponding to the image data performed the operation, to the head-mounted image display unit.
These and other objects and advantages of the present invention will become more apparent from the following detailed description.
According to the present invention, it is possible not only to easily confirm 3D images when the 3D images are created, but also to display the images on a large screen, and further, to provide an image observation apparatus having a small shape and excellent portability.
In addition, according to the present invention, it is possible to provide an image observation apparatus facilitating various settings and changes such as an aspect ration and picture quality when images of games and the like are displayed.
Furthermore, it is possible to provide an image observation apparatus facilitating various settings of the apparatus itself such as adjustment of a pupil distance and a parallax even if a head-mounted display is worn.
Still more, it is possible to provide an image observation apparatus not requiring to refer to an operation manual after wearing a display unit by making an image display part of the image observation apparatus display operation methods for a game to be started.
In addition, it is possible to provide an image observation apparatus capable of securely protecting the progress status of a game if power supply to the apparatus is interrupted when the apparatus is used for the game.